Policy Improvement for several Environments

In this paper we state a generalized form of the policy improvement algorithm for reinforcement learning. This new algorithm can be used to ...nd stochastic policies that optimize single-agent behavior for several environments and reinforcement functions simultaneously. We ...rst introduce a geometric interpretation of policy improvement, de...ne a framework to apply one policy to several environments, and propose the notion of balanced policies. Finally we explain the algorithm and present examples.